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DETAILED ACTION 

1 . This action is responsive to communications: Appeal Brief filed 1/29/2004, to the original application 
filed 2/6/2002, drawings filed 3/25/2002. The present application is a continuation (filed under 37 CFR 1.53(b)) 
of parent case 08/900,421 (now abandoned). Said parent case filed 7/25/1997. 

2. The examiner's rejections under 35 U.S.C. 103(a) set forth in the Final Office Action filed 9/10/2003 
(as paper 10), has been withdrawn. 

3. Claims 1, 3-20 are pending. Claims 1, 14, 16, 19, 20 are independent claims. 

4. In view of the Appeal Brief filed on 1/29/2004, PROSECUTION IS HEREBY REOPENED. A new 
ground(s) of rejection are set forth below. 

To avoid abandonment of the application, appellant must exercise one of the following two options: 

(1) file a reply under 37 CFR 1.111 (if this Office action is non-final) or a reply under 37 CFR 1. 1 13 (if 
this Office action is final); or, 

(2) request reinstatement of the appeal. 

If reinstatement of the appeal is requested, such request must be accompanied by a supplemental appeal 
brief, but no new amendments, affidavits (37 CFR 1 . 130, 1 . 13 1 or 1 . 132) or other evidence are permitted. See 
37 CFR 1.193(b)(2). 

Claim Rejections - 35 USC §103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 102 of this title, if the 
differences between the subject matter sought to be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which said subject matter pertains. Patentability shall 
not be negatived by the manner in which the invention was made. 
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6. Claims 1, 8-10, 12-13, 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Mendelson et al. (hereinafter Mendelson), U.S. Patent No. 5,754,783 issued May 1998. 

In regard to independent claim 1, Mendelson teaches interleaving (i.e. synchronizing) timed primary 
and untimed secondary data streams at a server, said resulting stream(s) eventually transported to clients for 
display (Mendelson Abstract, column 4 lines 18-22, column 8 lines 15-20). Mendelson defines timed primary 
content data as games, movies, audio music, etc., acquired and stored in a storage device (hard drive) on said 
server (Mendelson column 1 lines 13-22, column 6 lines 8-1 1, Figure 4 item 421). Mendelson defines untimed 
secondary content as video text, graphic images, private data such as files, databases, or stuffing bits, etc. 
(Mendelson column 1 lines 24-28, column 3 lines 50-54, column 6 lines 23-26, Figure 4 item 414). It is noted 
that any number of data streams can be utilized (Mendelson column 4 lines 54-58). 

Mendelson does not specifically disclose "motion" type data. However, Mendelson's secondary content 
(video text/images etc.) can at least be interpreted as suggestive of motion data since it is displayed as a 
video/movie type stream. In addition, Mendelson's secondary content can also comprise "padding", null/stuffing 
data, interleaved via a stuffing buffer, helping to preserve the quality (i.e. motion - wander, jitter, etc.) of the 
timed primary data (Mendelson column 2 lines 50-67, column 3 lines 9-20, 50-54, column 8 lines 14-17, Figure 
4 item 440). It would have been obvious to one of ordinary skill in the art at the time of the invention to interpret 
Mendelson's secondary data as a type of "motion" data, providing the benefit of controlling/minimizing possible 
timed motion related problems (i.e. wandering, jittering, etc.) of the primary content (compare the above two 
paragraphs with claim 1 "A method of synchronizing asynchronous time-based and motion data in a system in 
which the time-based data and the motion data are transmitted by a server over a network to a client, the 
method comprising", and "retrieving a time-based data stream and a motion data stream at the server... ") 

Mendelson teaches receiving various time-based and motion data at a server, as explained above. In 
addition, since Mendelson discloses typical timed and untimed content within a frames environment (Mendelson 



Application/Control Number: 10/068,829 Page 4 

Art Unit: 2176 

column 1 lines 17-18, 24-25), combined with Mendelson 's teaching of using the MPEG standard (typically 
applied to MPEG files, which are frame-based) (Mendelson column 4 lines 48-52), Mendelson's data streams 
can be interpreted as frame based (compare with claim 1 "... each stream comprising frames of data"). 

Mipndelson teaches variable buffering (at a server) by measuring the bit rate stream, and buffers via the 
use of variable usage (interleaving) of primary, secondary and stuffing content buffers (Mendelson column 6 
lines 62-67, column 8 lines 6-20, Figure 4 items 440, 450, 460). It is noted that each type of content has its own 
buffer, and are regulated accordingly. It is noted that Mendelson's resulting data stream can be interpreted as 
comprising at least two interleaved streams (now custom synchronized) of various data types. In addition, 
multiple elementary streams within a transport stream can be utilized (Mendelson column 4 lines 54-63, column 
6 lines 4-8) (compare with claim 1 "variable buffering one of the time-based data stream and the motion data 
stream at the server to produce two streams having synchronized frames"). 

Mipndelson teaches transport of a final (custom synchronized) stream(s) to customers (clients) for 
eventual playback (Mendelson column 6 lines 4-8, column 8 lines 10-12; compare with claim 1 "using the 
synchronized frames at the client for playback of synchronized motion and time-based data to a user. "). 

In regard to dependent claim 8, Mendelson teaches primary content (video, audio data, etc.) as well as 
secondary content (video text, images, files, database data, along with "stuffing" data, etc.) as explained in the 
rejection of claim 1. As the resulting interleaved stream(s) is played back on a client, any combination of 
foreground/background combination can be typically specified, (i.e. the primary data (i.e. a movie scene) can be 
run as background to the secondary/motion data controlling it, or run as background to any related database files 
in the foreground, etc). \ 
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In regard to dependent claims 9, 10, Mendelson teaches buffering/processing occurring at the same 
time the program is being transported to customer equipment (Mendelson column 8 lines 9-12; compare with 
claim 9). 

Mendelson teaches audio (i.e. voice) data (Mendelson column 4 lines 57-59; compare with claim 10). 

In regard to dependent claims 12, 13, Mendelson teaches buffering of streams comprising buffering of 
primary and secondary content (said primary content can be audio data, Mendelson column 4 lines 57-59), along 
with necessary interleaved motion type data (secondary/stuffing data), producing a sequenced synchronized 
stream regarding constant verses variable bit stream rate (see also rejection of claim 1, also Mendelson column 6 
lines 62-67). 

In regard to independent claim 16, claim 16 reflects the apparatus used to perform the methods 
comprising computer readable instructions as claimed in claim 1, and is rejected along the same rationale. 



7. Claim 3 is rejected under 35 U.S.C. 103(a) as being unpatentable Mendelson as applied to claim 1 
above, and further in view of Baumgartner et al. (hereinafter Baumgartner), U.S. Patent No. 5,642,171 
issued June 1997. . 

In regard to dependent claim 3, Mendelson teaches buffering based on a calculated difference (delay) 
between a variable bit stream, and a constant bit stream (Mendelson column 6 lines 62-67). Mendelson does not 
specifically teach a difference between the delay for motion and time-based data stream to determine buffering 
for a fester stream. However, Baumgartner teaches a method whereby a current video frame number is 
subtracted from a current audio frame number to determine if the audio is too far ahead of the video 
(Baumgartner column 13 lines 60-67, column 14 lines 1-2). It would have been obvious to one of ordinary skill 
in the art at the time of the invention to apply the difference calculation method of Baumgartner to Mendelson's 
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stream analysis, providing Mendelson the benefit of more accurate time synchronization to the server buffer 
method disclosed by Mendelson. 

8. Claims 4, 14-15, 18 is rejected under 35 ILS.C. 103(a) as being unpatentable over Mendelson, and 
further in view of Shibata, Y., Media synchronization protocols for packet audio-video system on 
multimedia information networks (hereinafter Shibata), IEEE; January 3-6, 1995 pp.594-601. 

In regard to dependent claim 4, Mendelson does not specifically teach transfer of data values for a 
frame occuring only for a frame that has changed since a last frame was transmitted. However, Shibata discloses 
a method whereby audio data is sent from the video server to the client station only during a "talk spurt", with 
constant frame rates occurring during periods of audio silence (Shibata p.597, section 4.3 "Silence detected 
synchronization"). It would have been obvious to one of ordinary skill in the art at the time of the invention to 
apply Shibata to Mendelson' s stream interleaving, providing Mendelson the benefit of file size and transmission 
reduction (streamlining) of the audio/video data streams. 

In regard to independent claim 14, claim 14 incorporates substantially similar subject matter as 
claimed in claim 1, and in further view of the following, is rejected along the same rationale. 

Mendelson teaches an embodiment using streams associated with "data channels" (Mendelson column 
6 lines l-8).Mendelson also teaches a header packet (a descriptor packet), said header typically used to describe 
data within transport stream data (i.e. each 188 byte packet) (Mendelson column 4 lines 47-53, Figure 2). 

Mendelson does not specifically teach transfer of data values (channels) for a occuring only for changed 
data values. However, Shibata discloses a method whereby audio data is sent from the video server to the client 
station only during a "talk spurt", with constant frame rates occurring during periods of audio silence (i.e. a talk 
spurt changes the data values) (Shibata p.597, section 4.3 "Silence detected synchronization"). It would have 
been obvious to one of ordinary skill in the art at the time of the invention to apply Shibata to Mendelson's 
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stream interleaving, providing Mendelson the benefit of file size and transmission reduction (streamlining) of 
the audio/video data streams. It is to be noted that in order to adjust frame rates as above, the server must know 
that a change in the data has occurred, which is typically achieved by comparing new data (the talk spurt) with 
the last recognized data value (previously recognized as empty - reflecting a period of silence). 

In regard to dependent claim 15, Mendelson teaches an embodiment using streams associated with 
data channels (Mendelson column 6 lines l-8).Mendelson also teaches a header packet (a descriptor packet), 
said header typically used to describe data within transport stream data (i.e. each 188 byte packet) (Mendelson 
column 4 lines 47-53, Figure 2). 

In regard to dependent claim 18, Mendelson teaches an embodiment using streams associated with 
"data channels" (Mendelson column 6 lines l-8).Mendelson also teaches a header packet (a descriptor packet), 
said header typically used to describe data within transport stream data (i.e. each 188 byte packet) (Mendelson 
column 4 lines 47-53, Figure 2). 

Mendelson does not specifically teach transfer of data values (channels) for a occuring only for changed 
data values. However, Shibata discloses a method whereby audio data is sent from the video server to the client 
station only during a "talk spurt", with constant frame rates occurring during periods of audio silence (i.e. a talk 
spurt changes the data values) (Shibata p.597, section 4.3 "Silence detected synchronization"). It would have 
been obvious to one of ordinary skill in the art at the time of the invention to apply Shibata to Mendelson's 
stream interleaving, providing Mendelson the benefit of file size and transmission reduction (streamlining) of 
the audio/video data streams. It is to be noted that in order to adjust frame rates as above, the server must know 
that a change in the data has occurred, which is typically achieved by comparing new data (the talk spurt) with 
the last recognized data value (previously recognized as empty - reflecting a period of silence). 
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9. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over Mendelson, as applied to 
claims 1, above, and further in view of Katseff et al. (hereinafter Katsefl), U.S. Patent No. 5,822,537 issued 
October 1998. 

In regard to dependent claim 5, Mendelson teaches a networking environment (Mendelson column 4 
lines 28-33). Mendelson does not specifically teach using the Internet for distribution. However, Katseff teaches 
an internet as an example of suggested types of networks that can be used (Katseff, column 3 lines 58-63). It 
would have been obvious to one of ordinary skill in the art at the time of the invention to apply Katseff to 
Mendelson, providing Mendelson the benefit of a widespread and familiar network. 



10. Claims 6, 7, 11, 17, 19 are rejected under 35 U.S.C 103(a) as being unpatentable over Mendelson, 
and further in view of Durward et al. (hereinafter Durward), U.S. Patent No. 9,950,202 issued September 
1999. 

In regard to dependent claim 6, Mendelson teaches secondary data as stuffing data, or as private data 
such as files or databases (see rejection of claim 1, also Mendelson column 6 lines 24-26). Mendelson does not 
specifically teach said motion data mapped to control movement of a virtual figure displayed in a scene at the 
client. However, Durward teaches a method whereby updated positional data from a person's head position 
sensor is mapped and used to determine the position of a virtual being defined for that user, communicating 
graphical data to the user via database data (Durward, column 6 lines 29-32, 46-52, column 7 lines 12-20). It 
would have been obvious to one of ordinary skill in the art at the time of the invention to apply the virtual 
position database data of Durward to Mendelson's secondary database data, providing Mendelson increased 
spatial imagery to the multimedia stream. 
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In regard to dependent claim 7, Mendelson does not specifically teach using a body suit for 
generating motion data. However, Durward teaches a method of sensing position data with the use of a body suit 
(Durward, column 3 line 25-33). It would have been obvious to one of ordinary skill in the art at the time of the 
invention to apply Durward to Mendelson, providing Mendelson the benefit of three dimensional physical 
contour mapping, providing increased spatial image accuracy to Mendelson's video stream method. 

In regard to dependent claim 11, Mendelson teaches an embodiment using streams associated with 
data channels (Mendelson column 6 lines l-8).Mendelson also teaches a header packet (a descriptor packet), 
said header typically used to describe data within transport stream data (i.e. each 188 byte packet) (Mendelson 
column 4 lines 47-53, Figure 2). 

Mendelson does not specifically teach that a client may join in progress a multicasting of a stream. 
However, Durward teaches virtual reality presentations where a number of contributing user's data is collected 
and analyzed accordingly, so a to facilitate multicasting (and interaction between figures) of the virtual world to 
various users (Durward Figures 4-6, 7 column 8 lines 5-15). Durward also teaches users logging on to a system 
via modems, said users data is shared with other users and continuously updated so as to create a virtual reality 
world (Durwald, column 8 lines 22-24, 41-64, Figure 1). It would have been obvious to one of ordinary skill in 
the art at the time of the invention to apply Durward to Mendelson, providing Mendelson the benefit of 
multicasting associated with synchronization of streams. 

In regard to dependent claim 17, Mendelson does not specifically teach an apparatus whereby 
synchronized motion/time-based data is milticasted to clients in a network. However, Durward teaches an 
apparatus whereby users log on to a system via modems, said users data is shared with other users and 
continuously updated so as to create a virtual reality world (Durward, column 8 lines 22-24, 41-64, Figure 1). It 
would have been obvious to one of ordinary skill in the art at the time of the invention to apply the data sharing 
method of Durward to Mendelson, providing Mendelson the benefit of continuous updating and increased 
uniform user awareness in a virtual environment. 
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In regard to independent claim 19, claim 19 incorporates substantially similar subject matter as 
claimed in claim 1, and in further view of the following, is rejected along the same rationale. 

Mendelson teaches secondary data as stuffing data, or as private data such as files or databases (see 
rejection of claim 1, also Mendelson column 6 lines 24-26). Mendelson does not specifically teach said motion 
data mapped to control movement of a virtual figure displayed in a scene at the client, along with 
synchronization. However, Durward teaches a method whereby updated positional data from a person's head 
position sensor is mapped and used to determine the position of a virtual being defined for that user, 
communicating graphical data to the user via database data (Durward, column 6 lines 29-32, 46-52, column 7 
lines 12-20). Since the playback of Mendelson's resulting stream synchronizes the secondary data with the 
primary content, the primary data (i.e. video) can be synchronized with added secondary database data (related 
to a virtual figure). It would have been obvious to one of ordinary skill in the art at the time of the invention to 
apply the virtual position database data of Durward to Mendelson's secondary database data, providing 
Mendelson increased spatial imagery and desired synchronization to the resulting multimedia stream. 

11. Claim 20 is rejected under 35 U.S.C. 103(a) as being unpatentable over Mendelson, Baumgartner, 
and Durward. 

In regard to independent claim 20, claim 20 incorporates substantially similar subject matter as 
claimed in claims 1, 3, 10, 1 1, and is rejected along the same rationale. 

Response to Arguments 

12. Applicant's arguments with respect to claims 1-20 have been considered but are moot in view of the new 
ground(s) of rejection. 



# 
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Conclusion 



13. Any inquiry concerning this communication or earlier communications from the examiner should be 
directed to William Bashore whose telephone number is (703) 308-5807. The examiner can normally be 
reached Monday through Friday from 1 1 :30 AM to 8:00 PM EST. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, Joseph 
Feild, can be reached on (703) 305-9792. 

Any inquiry of a general nature or relating to the status of this application should be directed to the 
Group receptionist whose telephone number is (703) 305-3900. 

14. Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D C. 20231 

or faxed tp: 

(703-872-9306) (for formal/after-final communications intended for entry) 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, 
Arlington, VA, Fourth Floor (Receptionist). 



William L. Bashore 
Patent Examiner, AU 2176 
April 16, 2004 



SUPERVISORY PATENT EXAMINER 




